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ABSTRACT 

Straightforward physical arguments are used to derive the properties of Lya forest absorbers. It is shown 
that many aspects of the current physical picture of the forest, in particular the fact that the absorption arises 
in extended structures of moderate overdensities which contain a large fraction of the baryons in the universe, 
can be derived directly from the observations without making any specific assumptions about the presence and 
distribution of dark matter, the values of the cosmological parameters or the mechanism for structure formation. 
The key argument is that along any line of sight intersecting a gravitationally confined gas cloud, the size of the 
region over which the density is of order the maximum density, is typically of order the local Jeans length. This is 
true for overdense absorbers, regardless of the shape of the cloud and regardless of whether the cloud as a whole 
is in dynamical equilibrium. The simple analytic model is used to derive the mass distribution of the photoionized 
gas directly from the observed column density distribution. It is demonstrated that the shape of the column density 
distribution, in particular the observed deviations from a single power-law, and its evolution with redshift, reflect 
the shape of the matter distribution and can be understood in terms of the growth of structure via gravitational 
instability in an expanding universe. 

Subject headings: cosmology: theory — galaxies: formation — intergalactic medium — quasars: absorption 
lines — hydrodynamics 



1. INTRODUCTION 

The Lya forest is thought to arise in the smoothly fluctuating 
intergalactic medium (IGM), which traces the distribution of 
the dark matter on large scales. Mildly overdense regions give 
rise to low column density absorption lines (Nhi <, 10 14 5 cm" 2 ), 
whose widths are determined mainly by thermal broadening 
and the differential Hubble flow across the absorber. Although 
many aspects of this physical picture of the Lya forest can al- 
ready be found in papers by Bi, Borner, & Chu (1992) and Bi 
(1993), who used a semi-analytic model with a log-normal dis- 
tribution for the dark matter; it gained acceptance only after 
it was confirmed and extended by hydrodynamical simulations 
(e.g., Cen et al. 1994; Zhang, Anninos, & Norman 1995; Hern- 
quist et al. 1996; Miralda-Escude et al. 1996; Theuns et al. 
1998). In the simulations the Lya forest arises in a network 
of sheets, filaments, and halos, which give rise to absorption 
lines of progressively higher column densities. This physical 
picture of the forest is supported by observations of gravita- 
tionally lensed quasars and quasar pairs, which show that the 
characteristic sizes of the absorbers are 10s to 100s of kpc (e.g., 
Bechtold et al. 1994; Dinshaw et al. 1994, 1995; Smette et al. 
1995) and by the success of cosmological simulations in re- 
producing the statistics of the observed absorption spectra (see 
Efstathiou, Schaye, & Theuns 2000 for a recent review). 

Following Rees (1986), most theoretical work on the Lya 
forest has been carried out within the framework of the cold 
dark matter paradigm. In virtually every case an ab-initio ap- 
proach was followed, i.e. the properties of the forest were pre- 
dicted for a particular model of structure formation and then 
compared to observations (but see for example Weinberg et al. 
1997 and Nusser & Haehnelt 1999 for notable exceptions). The 
models, which can be either semi-analytic or fully numerical, 
are specified by the cosmological parameters, the initial power 
spectrum, and the evolution of the UV background radiation. 



In this paper a rather different approach will be taken. It will 
be shown that many properties of the absorbers, including their 
densities and radial sizes, can be derived using straightforward 
physical arguments, independent of the details of the cosmo- 
logical model. The key physical argument is that the absorbers 
will generally not be far from local hydrostatic equilibrium, i.e. 
the size of the region over which the density is of order the 
maximum density along the sightline, is of order the local Jeans 
length. It will be argued that this is very likely the case for over- 
dense absorbers, regardless of whether the cloud as a whole is 
in hydrostatic equilibrium. The same formalism will be used to 
explain the observed structure in the column density distribu- 
tion and to derive the density distribution of the photoionized 
IGM directly from the observations. 

The main physical arguments underlying this work are pre- 
sented in section ||. Although the expressions derived in section 
|2] apply to all gravitationally confined gas clouds, only pho- 
toionized, optically thin clouds (Nhi <J 10 17 cm" 2 ) are discussed 
in subsequent sections. The properties of self-shielded clouds, 
which are relevant for protogalactic clumps, Lyman limit sys- 
tems, and damped Lya absorbers, are derived in Schaye (2001). 
The formalism of section^ is used to derive the physical proper- 
ties of Lya forest absorbers in section|3| The cosmological im- 
plications of the observed column density distribution are dis- 
cussed in section [I| Finally, the main conclusions are summa- 
rized in section & Throughout the paper, numerical values will 
be inserted for physical parameters whose values are thought 
to be well known. The full expressions can be found in the 
appendix. 

2. HYDRODYNAMICS 

In this section the physical properties of self-gravitating gas 
clouds will be derived. It will be argued that along each 
sightline through an overdense absorber, the sound crossing 
timescale f sc , is of order the local dynamical timescale ?d yn . In 
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other words, the characteristic size of the absorber is typically 
of order the local Jeans length. The characteristic size is de- 
fined as the size of the region for which the density is of order 
the characteristic density of the absorber. In section 2.1 it will 
be shown that because most of the absorption takes place in 
the densest gas along the sightline, a well-defined characteristic 
density does exist. 

Consider a cloud with characteristic density n#. The dynam- 
ical (or free-fall) time is 
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where f g is fraction of the mass in gas 1 and Y is the baryonic 
mass fraction in helium, which will be set equal to 0.24 here- 
after. In cold, collapsed clumps f g could be close to unity, but 
on the scales of interest here f g will not be far from its universal 
value, f g w Q,b /Cl m . Let L be the characteristic size of the cloud, 
i.e. the length of the intersected part of the cloud over which the 
density is of order the characteristic density. The sound cross- 
ing time is then 



t K = 2.0 x 10 1 
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where c s is the sound speed in an ideal, monatomic gas with 
the ratio of specific heats 7 = 5/3, T = T4 x 10 4 K, and p, is 
the mean molecular weight. In what follows /1 will be set equal 
to the value appropriate for a fully ionized, primordial plasma, 
= 4/(8-57) wO.59. 

The equation for hydrostatic equilibrium is dP/dr = 
-GpM/r 2 , where M is the mass interior to r. Since the pres- 
sure P ~ c 2 s p, this implies c 2 s p/L ~ Gp 2 L, i.e. f sc ~ fd yn - There 
are many other ways of showing that in hydrostatic equilibrium 
?sc ~ ?dyn- It can for example readily be seen that it implies that 
the sound speed is of order the circular velocity. The condi- 
tion f sc = fdyn defines a characteristic length, the so-called Jeans 
length 2 , 
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To make a connection with observations, it is useful to intro- 
duce the 'Jeans column density', 
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If f sc ^> fdyn, then the cloud is Jeans unstable and will either 
fragment or, since v ~ L/fd yn ^S> c s , shock to the virial tempera- 
ture. In either case, equilibrium will be restored on the dynam- 
ical timescale. If, on the other hand, f sc <C fdyn, then the cloud 
will expand or evaporate and equilibrium will be restored on 
the sound crossing timescale. 

Although the Jeans criterion for gravitational instability is 
familiar, it is not always appreciated that a growing density per- 
turbation will generally be close to 'local hydrostatic equilib- 
rium', i.e. t sc ~ fdyn locally, but not necessarily for the cloud as 
a whole. In other words, along any sightline through the evolv- 
ing cloud, which will in general not be spherical, the length 

1 Stars and molecules do not contribute to f g . 

2 The Jeans length or, equivalently, the dynamical timescale is not well defined and some authors include a dimensionless coefficient of order unity in the definition. 
However, the value of the dimensionless coefficient depends on the geometry of the cloud and it is hard to justify any particular choice. 



of the region for which the density is of order the maximum 
density, will be of order the local Jeans length. If the cloud has 
substructure, then the condition of local hydrostatic equilibrium 
should be applied to each density maximum along the sightline. 
Note that the concept of a Jeans mass only makes sense if the 
cloud is roughly spherical. The collapse will in general not be 
synchronized along all three spatial dimensions, and a density 
perturbation may first collapse into a sheet-like or filamentary 
structure before reaching a more spheroidal configuration. Fur- 
thermore, the tidal field around collapsed regions can induce 
the formation of a network of sheets and filaments. Although 
the mass of the region containing the perturbation can thus dif- 
fer substantially from the Jeans mass, along any sightline in- 
tersecting an evolving perturbation, the characteristic size will 
generally be of order the local Jeans length. 

Large departures from local hydrostatic equilibrium do occur 
when the pressure changes on a timescale much shorter than 
fdyn- For example, during shell crossing the density suddenly 
increases by a large factor. This will result in shocks, leading to 
virialization and equilibrium will quickly be restored. A sudden 
drop in the temperature through a thermal instability is another 
example. In that case equilibrium will be restored through frag- 
mentation. These two examples are relevant for high column 
density absorbers and are discussed in more detail in Schaye 
(2001). For the Lya forest, sudden heating is in fact more rele- 
vant. During reionization the temperature of the diffuse IGM is 
suddenly raised by several orders of magnitude to at least 10 4 K 
and possibly ;> 10 5 K, depending on the speed of reionization 
and the spectrum of the ionizing radiation. Because of the sud- 
den increase in the Jeans length during reionization, f sc fd yn 
for most of the gas. Until pressure forces have had time to re- 
store local hydrostatic equilibrium, the gas can be clumpy on 
scales smaller than the Jeans length (e.g., Gnedin & Hui 1998). 
Simulations by Gnedin (2000) show that even if the IGM was 
reionized as late as z ~ 7, the baryon smoothing scale is again 
similar to the Jeans scale by the redshifts of interest here (z < 4). 

Pressure forces can only smooth the gas distribution on scales 
smaller than the sound horizon, which is defined by the con- 
dition f sc ~ H~ l . Hydrostatic equilibrium would then require 
fdyn ~ H~ l , which implies p ~ p. Hence, the assumption of lo- 
cal hydrostatic equilibrium may break down for absorbers with 
characteristic densities smaller than the cosmic mean. The pre- 
cise value of the sound horizon depends on the thermal his- 
tory of the IGM. After reionization, the temperature of the low- 
density IGM is expected to drop. At very high redshift (z ^> 7) 
the IGM can cool efficiently through inverse Compton scatter- 
ing off the microwave background, but after that the main cool- 
ing mechanism is adiabatic expansion and the cooling time is 
thus of order the Hubble time. Hence, for z < z re ion, the sound 
horizon is greater than c s /H, which implies that the statement 
that local hydrostatic equilibrium is only a good approximation 
for overdense absorbers, is conservative. 

Finally, if a cloud is not gravitationally supported, then local 
hydrostatic equilibrium does not necessarily imply that its char- 
acteristic size is of order the local Jeans length. A cloud that is 
not supported by self-gravity can be pressure confined, rota- 
tionally supported or it can be not supported at all. Although 
pressure confinement may be important for the high column 
density absorption lines, the possibility that a significant frac- 
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tion of the Lya forest arises in pressure confined clouds has 
been ruled out on both observational and theoretical grounds 
(e.g., Rauch 1998). Even if every Lya absorption line were to 
arise in a rotationally supported cloud, then the characteristic 
size will only be much greater than the Jeans length for sight- 
lines that are nearly perpendicular to the spin axis of the cloud, 
which cannot be true for a large fraction of the absorption lines. 
As discussed above, unless the cloud is kept in a steady state by 
external pressure or by a centrifugal barrier, local hydrostatic 
equilibrium is restored on a timescale t <C H~ l if the absorber 
is overdense. Hence, if a population of Lya forest absorbers 
exists that are quasi-stable, i.e. whose lifetimes are not much 
shorter than the Hubble time, then any given absorption line is 
unlikely to arise in a transient cloud, i.e. a cloud with a lifetime 
t <C H~ l , unless most of the matter in the universe is contained 
in clouds far from dynamical equilibrium. In summary, it is 
very likely that local hydrostatic equilibrium is a good approx- 
imation for most Lya forest absorbers. 

2.1. The characteristic density 

The characteristic density is the column density weighted 
density of an absorber. It is easily demonstrated that for a wide 
range of density profiles, the neutral hydrogen column density 
through a cloud is dominated by the region with the highest 
density. For a spherical cloud with a density profile oc r~", 
the neutral hydrogen column density in a sightline with impact 
parameter b is 



N HI cc 



(b 2 + l 2 T n ld\nl, 



(5) 



where I used the fact that the neutral fraction in an optically 
thin gas is proportional to the density. For n > 1/2, the dom- 
inant contribution to the column density comes from / < b 
and Nhi ~ nni{b)b. For example, for a singular isothermal 
sphere (n = 2), the column density weighted density is (n#) = 
/ nnriHidl/ J nnidl = \nn(V). Hence, for reasonable density 
profiles, almost all the absorption takes place in the densest part 
of the gas cloud. This implies the existence of a characteristic 
density, which will be close to the maximum density along the 
line of sight through the cloud. 

3. PHYSICAL PROPERTIES 

In the previous section expressions were derived for the size 
and total hydrogen column density of an absorber as a function 
of its density and temperature. To make contact with observa- 
tions it is necessary to compute the neutral fraction since it is the 
neutral hydrogen column density that is observed. Here, only 
optically thin clouds (Nhi <, 10 17 cm" 2 ) will be considered, for 
which the ionization correction can be computed analytically. 

The neutral fraction of a highly ionized, optically thin gas is 



n H i 



■■nePmiT- 1 ~ 0.46 nnT^T^ 



(6) 



where fan «4x lO" 13 ^" ' 76 em's" 1 and T = Ti 2 x 1(T 12 s" 1 
are the hydrogen recombination and photoionization rates re- 
spectively and n e is the number density of free electrons. The 
photoionization rate corresponding to a UV background inten- 
sity of J(v) erg s" 1 cm" 2 sr" 1 Hz" 1 is 



r = 



4irJ(v)a(v) 
hv 



dv, 



(7) 



The cooling time depends on the neutral fraction and thus on Y. If Fi2 "C 1 
lower densities than l(T 4 cm~ 3 . 



where a is the cross section for photoionization and vi is the 
frequency at the Lyman limit. At redshifts 2 < z <J 4, studies 

of the proximity effect find T ~ 10~ 12 s" 1 with a relative uncer- 
tainty of order unity (Scott et al. 2000 and references therein), 
which is in reasonable agreement with models of the UV back- 
ground from quasars (e.g., Haardt & Madau 1996). At low red- 
shift (z < 0.5) the intensity of the UV background is thought to 
be considerably lower, V « 10~ 14 - 10~ 13 s" 1 (e.g., Dave & Tripp 
2001 and references therein). 

To make further progress, an estimate of the temperature 
is needed. It has been known since long that the widths of 
Lya absorption lines are consistent with photoionization tem- 
peratures, T ~ 10 4 K (e.g., Carswell et al. 1984). Gas with a 
temperature significantly lower than this is heated on the pho- 
toionization timescale, t = T~ l <C H~ l . To reach temperatures 
T ^> 10 4 K, the gas would have to be shock-heated. Accretion 
shocks associated with gravitational collapse are likely to be 
important only for gas at high overdensities, but galactic winds 
could in principle heat a large fraction of the IGM. For densi- 
ties tih ;> lO^cm -3 (6 ;> 10 at z ~ 3), atomic cooling is effi- 
cient 3 and the temperature will be approximately 10 4 K (molec- 
ular cooling can lead to lower temperatures, but the molecular 
fraction is negligible for optically thin clouds). Cooling can be- 
come inefficient if the temperature is very high, T ^> 10 5 K, but 
in that case collisional ionization suppresses the neutral fraction 
and the gas will only give rise to lines of very low column den- 
sity. In short, T ~ 10 4 is likely to be a good approximation for 
the Lya forest, although a somewhat higher temperature could 
be appropriate at n# <; 10~ 4 cm" 3 if galactic winds are impor- 
tant. 

3.1. Densities 

Substituting equation || into equation Q yields a relation be- 
tween the neutral hydrogen column density and the characteris- 
tic density, 
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Because the gas responsible for the low-column density Lya 
forest is still expanding, it is often instructive to use the density 
contrast, S = (nH-h~H)/h~H, where 



mi ~ 1.1 x 10 cm 



5 ™-3 



i+z\ 3 fn b h 2 



4 / V0.02 



(9) 



instead of n#, 



Nhi - 2.7 x 10 13 cm -2 (l+d) 3 ^ 126 ^ 



l+z 
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Clearly, the low-column density Lya forest arises in gas of 
low overdensity. Virialized minihalos have 6 > 10 2 which 
corresponds to high column density absorption lines, Nhi > 
10 16 cm -2 at z ~ 3. The presence of dark matter does not have 

as is the case at low redshift, then atomic cooling will become efficient at much 
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a dramatic effect on the inferred densities: if f g = 1 instead of 
0.16 (w f2fc/O m ), then the density contrast corresponding to a 
fixed column density is only about a factor 2 smaller. 

The neutral hydrogen column density corresponding to a 
fixed density contrast is about a factor 5 x 10 2 T(z = Q)/T(z = 
3) ~ 5-50 lower at z ~ than at z ~ 3, with only a weak depen- 
dence on the evolution of the temperature. Because the strong 
dependence on redshift is partially offset by the evolution of 
r, the local Lya forest is not fundamentally different from the 
z ~ 3 forest. 

Hydrodynamical simulations have been used to make de- 
tailed predictions for the low column density Lya forest. Al- 
though the formalism developed here does not make any spe- 
cific assumptions about the cosmology or the nature of the 
dark matter, it can of course be applied to the specific mod- 
els tested with the simulations. In the simulations (which 
generally do not include feedback from star formation or ra- 
diative transfer effects) it is found that the low-density gas 
(tin < 10~ 4 cm" 3 , (5 <; 10 at z ~ 3) follows a tight temperature- 
density relation, which can be approximated by a power-law, 
T = 7b(l + 6) a (Hui & Gnedin 1997). Substituting this relation 
into equation [H] yields, 
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From comparisons of the line widths in simulated and observed 
spectra it is found that 7b ~ 2 x 10 4 K and a ~ 0.0-0.3 at z ~ 3 
(e.g., Schaye et al. 2000; McDonald et al. 2001). 

Dave et al. (1999, hereafter DHKW) find that the following 
empirical relation fits the results of their simulation over the 

range z = 0-3: N H i ~ 10 14 cm" 2 [^(1 + 5)10 4z ] , although 
the scatter is large for z <; 1 . This relation is of course specific 
to the simulation of DHKW, i.e. it would have been different 
if they had used a simulation with a different cosmology, UV 
background or temperature. Nevertheless, we can compare it 
to equation [H] if we set all the other parameters to the values 
of DHKW's simulation 4 . The normalization (Nhi(6 = 0)) of 
DHKW turns out to be about a factor 3 higher (7.2 x 10 13 vs. 
2.3 x 10 13 ), which is about as good as could be expected consid- 
ering that there is an uncertainty of order unity in the normaliza- 
tion of both equation [H] (because of some arbitrariness in the 
value of the dimensionless coefficient in the definition of the 
Jeans length) and the empirical relation of DHKW (because of 
the arbitrariness in the definition of the density contrast corre- 
sponding to an absorption line in a simulated spectrum). In fact, 
the normalizations found by different groups vary by about an 
order of magnitude, with most groups finding somewhat lower 
values for N m (6 = 0) than DHKW (e.g., Bryan et al. 1999; 
Schaye et al. 1999). Equation |i"T| also reproduces the scaling of 
N m with 6 and z remarkably well. DHKW find N m oc (1 +6) 1 43 , 
whereas equation 11 gives Nhi oc (l+<5) 1,37 for the value of 
a found by DHKW (a w 0.5). In the simulation of DHKW, 
N H i(z = 3)/N HI (z = 2) » 3.7 and N HI (z = 2)/N HI (z = 0) « 6.3, 
while equation 11 yields values of 5.1 and 4.1, assuming that 
r evolves according to the model of Haard & Madau (1996), 
which was used by DHKW, and a constant temperature. In 

4 The simulation of DHKW has n b h 2 = 0.02, Q b /fZ„, = 0. 12 (= fy% (T 0A , a) ', 
of the ionizing background, but multiply the ionization rate by a factor 1.34. 



summary, equation [Q] matches the results produced by hydro- 
dynamical simulations very well. 

3.2. Radial sizes 

The radial sizes of absorbers can be expressed as a function 
of their neutral hydrogen column densities by combining equa- 
tions |, |, and §, 



L~ 1.0 x 10 2 kpc 



Nh 



10 1 



-1/3 , / f \2/3 



T 0.41 p-1/3 
M 1 12 



0.16 



(12) 

Hence, Lya forest absorbers are large, L ~ 10 2 kpc, with only a 
weak dependence on the column density, temperature, and ion- 
izing background. Even clouds of Nhi ~ 10 17 cm" 2 , the highest 
column density for which self-shielding can be neglected, have 
a characteristic size of 10 kpc. 

Equation [l2| shows that the physical size of an absorber of 
a fixed neutral hydrogen column density does not depend ex- 
plicitly on redshift. The main time dependence comes from 
the dependence on the ionization rate, which is thought to peak 
in the range z ~ 2-3. From redshift z ~ 3 to the ionization 
rate probably decreases by a factor 10-30, which implies that 
the physical sizes of z ~ absorbers are about a factor 2 to 3 
greater than those of z ~ 3 absorbers with the same Nhi- The 
comoving sizes, however, are smaller at z ~ than at z ~ 3 by 
a factor 4/3 to 2. 

If the absorber has not decoupled from the Hubble expansion, 
then the differential Hubble flow across the absorber broadens 
the absorption line, leading to a Hubble width of bn ~ H(z)L/2, 
where H is the Hubble parameter. The Hubble width is compa- 
rable to the thermal width, bj = \J 2kT / m » 12.8 T X J 2 kms" 1 , 
for Nhi ~ 10 13 — 10 14 cm" 2 . In reality, absorbers are not expand- 
ing freely and peculiar velocity gradients will change the red- 
shift space sizes of the absorbers. Note, however, that peculiar 
velocity gradients cannot decrease the width of a line below the 
thermal width. The minimum line width as a function of the 
column density can therefore be used to measure the temper- 
ature as a function of the density (Schaye et al. 1999; Ricotti, 
Gnedin, & Shull 2000; Bryan & Machacek 2000). Finally, it is 
worth noting that the Hubble width increases almost as rapidly 
with temperature as the thermal broadening width, bn oc T 0Al 
compared to bj oc T 5 . 

It is easily demonstrated that bn ~ bj for a density contrast 
6^0: since c s ~ bj and fd yn ~ 1 /H, hydrostatic equilibrium 
implies bn ~ HL ~ Hc s /td yn ~ bj. Similarly, it follows that for 
underdense absorbers c s < bn, i.e. sound waves cannot over- 
come the differential Hubble expansion across the absorber. 
This is just another way of showing that the fundamental as- 
sumption underlying this work, namely that the gas clouds will 
typically not be far from local hydrostatic equilibrium, breaks 
down for underdense absorbers. 

Equations [l(] and [j~2| predict the existence of a population 
of low column density (Nhi <J 10 13 cm" 2 at z ~ 3) absorp- 
tion lines that arise in underdense gas and are predominantly 
Hubble broadened. They should be broad and shallow and 
their profiles should deviate substantially from thermal profiles. 
Their characteristic sizes are greater than the sound horizon 
(Av ~ 10 2 kms" 1 ), which implies that the formalism developed 
in this paper is not well-suited to describe them. These lines 
trace the underlying matter distribution and can be regarded as a 

i (1.0,0.5) at z = 3. They use the model of Haardt & Madau (1996) for the evolution 



Schaye 



5 



"fluctuating Gunn-Peterson effect". This means that large-scale 
correlations in the Lya absorption can be used to constrain the 
primordial power spectrum (Croft et al. 1998). 

3.3. Masses 

If the absorbers are spherical, then their characteristic mass 
is Mj = pi} . Hence, the characteristic mass in gas is, 



M„ ~ 8.8 x 10" M 



N, 



in 



10' 



-1/3 _ / , x5/3 



^1.41^-1/3 
12 



0.16 



(13) 

and the total mass is a factor l/f g higher. Of course, the ab- 
sorbers may very well be far from spherical, in which case they 
can have masses very different from equation [l3|. Indeed, one 
would not expect mildly overdense regions, S <J 10, to be spher- 
ical, as the collapse will generally not be synchronized along 
all three spatial dimensions. According to equation |l0|, such 
regions will give rise to absorption lines with column densities 
N H ,< 10 15 cm" 2 atz-3. 

4. COSMOLOGICAL IMPLICATIONS 

From the observed incidence of absorption lines one can 
compute the total mass in neutral hydrogen, Hhi, by integrating 
over the column density distribution. Similarly, if the ioniza- 
tion correction is known, then the total baryon density in pho- 
toionized gas can be computed. Furthermore, the distribution 
of mass as a function of density can be derived from the differ- 
ential column density distribution. The results obtained in the 
previous section make it possible to do this since equations ^ 
and U can be combined to yield the neutral fraction as a func- 
tion of the neutral hydrogen column density. 

4.1. Contribution to f2j 

The mean gas density relative to the critical density can be 
obtained from the neutral hydrogen column density distribu- 
tion, 

^ = / ^-f(^,z)dN HI , (14) 

3H ( _)c(l-Y)J n H i 

where f(NHi,z) is the number density of absorption lines per 
unit absorption distance X (a dimensionless quantity defined by 
equation |l5| below), and per unit column density. Observational 
results are usually expressed in terms of this function. Its rela- 
tion to the quantity that is actually observed, the number density 
per unit redshift, dn/dz, depends on the assumed cosmology: 



f(N HIl z) = 



a In 



d 2 n H(z) 1 



dN HI dX dNmdz H (l+z) 2 ' 
Combining equations pi M and [14] yields 



(15) 



fi s ~2.2x 10" 9 /i _1 r|f 



0.16 



1/3 



r 4 059 j N x Jj f{N H i,z}dNm, 
(16) 

where h = //o/(100 kms 1 Mpc '). If the gas is not isothermal, 
then T will depend on Nhi and should therefore be moved in- 
side the integral. Note that if f g = Qb/Cl m , then equation [If] 
agrees with the scaling relation familiar from the fluctuating 
Gunn-Peterson approximation, f2 ? cx r 1 / 2 /;" 3 / 2 (e.g., Rauch et 
al. 1997). 

Observations indicate that at high redshift the column den- 
sity distribution can be fit by a power-law, / = BN^j with 



f3 ~ 1.5, over a wide range of column densities (Tytler 1987). 
The fact that a single power-law appears to be a reasonable 
approximation is clear from Figure 1, which shows the ob- 
served distribution at z ~ 2.8. Hu et al. (1995) find that their 
data (solid points in Figure 1) is best fitted by a power-law 
with B = 5.3 x 10 7 and /3 = 1.46 (for (fi m ,0 A ) = (0.3,0.7)), and 
similar values are reported by others (e.g., Kirkman & Tytler 
1997). This fit is shown as the dashed line in Figure 1. As- 
suming (T4,Ti2,f g ,h) = (2.0, 1.0,0.16,0.65), equation |l6| gives 
ttgh 2 (logN HI = 13-17.2) « 0.015. Comparing this to the value 
derived from high redshift observations of the deuterium abun- 
dance, fL b h 2 w 0.020 (O'Meara et al. 2001), we see that this 
leaves little room for stars, collisionally ionized gas, and gas in 
absorbers with logNni < 13 or logNni > 17.2. Clearly, a large 
fraction of the baryons reside in the Lya forest. 

It is important to note that it was implicitly assumed that the 
measured column densities of the absorption lines are similar 
to the true column densities. This assumption may, however, 
be incorrect for low column density lines. Observationally, the 
column density distribution is determined by decomposing ab- 
sorption spectra into sets of Voigt profiles. In high-resolution 
spectra with a high signal-to-noise ratio, multiple Voigt com- 
ponents are usually required to obtain a satisfactory fit for a 
single absorption line. Low column density components are of- 
ten placed in the wings of absorption lines. If these spurious 
lines were to dominate the column density distribution below 
some value of Nhi, then it would not be possible to use that part 
of the distribution to estimate the baryon density. 

Another, related problem is that Monte Carlo simulations 
are often used to "correct" the observed column density dis- 
tributions for the fraction of low column density lines that are 
thought to be "missing" because they are heavily blended with 
stronger lines. This procedure can result in an unphysical ex- 
tension of the power-law distribution to arbitrary low column 
densities. In high quality spectra these corrections become im- 
portant below Nhi ~ 10 13 cm" 2 at z ~ 3 (e.g., Hu et al. 1995). 
The data points shown in Figure 1 indicate the observed dis- 
tribution, the "corrected" Hu et al. points would all fall on the 
dashed line. Unless current estimates of the baryon density are 
significantly in error, the distribution of column densities of real 
absorption lines, i.e. excluding Voigt components in the wings 
of absorption lines, cannot extend to much lower column den- 
sities. 

Finally, it has to be kept in mind that the value obtained for 
n g is uncertain to a factor of order unity because of the uncer- 
tainties in the values assumed for the various parameters and 
because of the fact that there is some arbitrariness in the value 
of the dimensionless coefficient in the definition of the Jeans 
length. These uncertainties do, however, not affect the derived 
scaling relations and the shape of the mass distribution, which 
is the subject of the next section. 



4.2. Mass distribution 

It is interesting to look at the differential distribution of 
the baryon density. If f(Nni,z) <x A^f, then equation [l6| im- 
plies n b cx J N H ^~ P d\nN m - Hence, if j3 < 4/3, then the 
mass per decade of column density is an increasing function 
of Nhi and similarly, since dhiNni cx dlnnn (equation |8]), 
the mass per decade of density is an increasing function of 
«h- There is evidence for deviations from a single power-law, 
the column density distribution seems to steepen somewhat 
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FIG. 1. — Observed neutral hydrogen column density distribution. Solid 
points are from Hu et al. (1995), not "corrected" for incompleteness; open cir- 
cles are taken from the compilation by Petitjean et al. (1993). The dashed line 
is the power-law fit given by Hu et al., which is the best fit to their "corrected" 
data in the regime 2 X 10 12 -3 X 10 14 cm" 2 , f(Nm) = 5.3 X 10 7 N H \ M (the 
normalization has been modified to that corresponding to a LCDM universe, 
(f!,„, Qa) = (0.3,0.7)). The mean redshift of both samples is 2.8. 

above 10 14 cm" 2 (Kim et al. 1997) before flattening to j3 w 1 .32 
at Nhi <; 10 16 cm" 2 (Petitjean et al. 1993). These trends are vis- 
ible in Figure 1 , although they are somewhat difficult to see in 
a log-log plot of /(Nhi) (they would have been more apparent 
if, contrary to convention, the quantity d \ '^ Hldx had been plot- 
ted instead of d ^ H " dx )- Although the reality of the deviations 
from the single power-law is well established, their physical 
origin is not understood. In this section it will be shown that 
the shape of the column density distribution reflects the shape 
of the mass-weighted probability density distribution for the 
gas density, i.e. the mass distribution of the gas as a function of 
its density. 

The mass distribution of the photoionized gas, dQ, g / d\og( \ + 
5), can be derived from the observed column density distribu- 
tion using equations [l(] and [l(| The results for the data points 
shown in Figure 1 are plotted in Figure 2. The data points 
were computed using the same parameter values as were used 
in the previous section. Points corresponding to underdense 
gas (6 < 0) are less reliable because the assumption of local 
hydrostatic equilibrium may break down, as was discussed be- 
fore. Furthermore, the corresponding column densities are so 
low (Nhi < 10 13 cm" 2 , column densities are indicated on the 
top axis of Figure 2) that many lines may have been lost in the 
noise. The two highest density data points are less reliable be- 
cause the effect of self-shielding is no longer negligible. From 
equation |l^ it can be seen that all points are proportional to the 
quantity h^T 1 ^ fg T^' 59 . Changing this quantity (or adding a 
dimensionless coefficient in the definition of the Jeans length) 
is equivalent to shifting the distribution vertically, but does not 
change the shape of the distribution. 

In calculating dVL g /d\og{ \ + S) from f(Nni), both the gas 
fraction f g and the temperature T were assumed to be inde- 
pendent of Nhi- Hence, the shape of the distribution shown in 
Figure 2 is determined by the shape of the column density dis- 
tribution. If the latter had been a strict power-law with spectral 
index /?, then the former would also have been a power-law, 
but with index 4/3-/3. Hence, the shape of the matter distribu- 
tion function traces the deviations from the single power-law in 



FIG. 2. — Mass distribution of photoionized gas. Data points, are com- 
puted from the observed column density distribution and equation lift using the 
same parameter values as were used in section ft. If (T$,T {2,fg,h^Mm,£lA.) = 
(2.0, 1.0,0.16,0.65,0.3,0.7). The solid line indicates the mass distribution of 
all the gas in a hydrodynamical simulation at z = 2.75, the dashed line denotes 
the distribution of the gas that has a temperature log T < 4.8. The curves have 
been normalized to have the same maximum as the Hu et al. (1995) points. The 
shape of the derived mass distribution suggests that gravitational instability is 
responsible for the growth of structure. The fall-off at <5 ~ 10 (log Nhi ~ 14.5) 
can then be attributed to the onset of rapid, non-linear collapse and the flatten- 
ing at 5 ~ 10 2 (logNut ~ 10 16 ) to virialization. The shape of the distribution 
agrees well with the shape of the gas distribution in the hydrodynamical simu- 
lation (solid curve), in particular if hot gas, for which collisional ionization is 
important, is excluded (dashed curve). 

Figure 1 . The turnover at Nhi <J 10 13 cm" 2 may thus be real and 
be caused by the turnover of the density distribution below the 
mean density. However, as discussed before, both the observa- 
tions and the physical model are unreliable in this regime. The 
steepening at Nhi ~ 10 14 5 cm" 2 reflects the fall-off in the den- 
sity distribution due to the onset of rapid, non-linear collapse. 
Finally, the flattening at Nhi <; 10 16 cm" 2 can be attributed to 
the flattening of the density distribution at 5 ;> 10 2 due to the 
virialization of collapsed matter. 

To show that the derived matter distribution is consistent with 
what would be expected from gravitational instability, the dis- 
tribution of gas in a hydrodynamical simulation at z = 2.75 is 
indicated by the solid line in Figure 2. The simulation, which 
was kindly provided by T. Theuns, has the same cosmologi- 
cal parameters as were used to calculate the data points. It has 
the same ionization background as the simulation described in 
Schaye et al. (2000), and the temperature of the low-density gas 
agrees with the measurements of those authors. To compare the 
shape of the matter distribution functions, the simulation data 
has been shifted vertically so that it has the same maximum as 
the Hu et al. (1995) points. 

The shape of the simulated matter distribution agrees fairly 
well with the distribution derived from the observations, al- 
though it seems to overpredict the mass fraction in the range 
5 ~ 10-100. Most of this excess can in fact be attributed to 
hot gas for which collisional ionization is important. This gas 
is much more highly ionized than cooler, photoionized gas of 
the same density and will therefore give rise to absorption lines 
of much lower column densities than predicted by equation [lfj. 
The dashed curve indicates the density distribution of gas cooler 
than 10 4 8 K, a temperature sufficiently low for collisional ion- 
ization to be negligible. The agreement between this distribu- 
tion and the points derived from the column density distribution 
of Petitjean et al. (1993) is remarkable. 

The slope of the column density distribution over the range 
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Nhi ~ 10 —10 cm is found to steepen from z ~ 2 to z ~ 
(e.g., Dave & Tripp 2001; Penton, Shull, & Stocke 2000; Kim, 
Cristiani, & D'Odorico 2001; Weymann et al. 1998). If gravita- 
tional instability is responsible for the shape of the gas density 
distribution, then this distribution will be similar at z ~ and 
z ~ 3, provided it is expressed as a function of the density rel- 
ative to the cosmic mean. Hence, the change in the slope of 
the column density distribution must mainly be due to the evo- 
lution of the function Nhi(S), which is determined by the ex- 
pansion of the universe and the evolution of the ionizing back- 
ground. Equation [h] shows that changes in z or T are equivalent 
to shifting the A^///-axis (the upper x-axis) in Figure 2 relative 
to the (1 + <5)-axis (the lower x-axis). 

In section 3. 1 it was shown that the neutral hydrogen column 
density corresponding to a fixed density contrast is about a fac- 
tor 5 x 10 2 r(z = 0) /T(z = 3) ~ 5-50 lower at z ~ than at z ~ 3. 
At z ~ 3, the distribution is found to be steepest in the range 
logNHi ~ 14.5-15.5, which corresponds to 6 ~ 5-25. This 
same range in density contrast corresponds to logNni ~ 13.2- 
14.2 at z ~ 0. Hence, the steepening of the slope of the column 
density distribution has the same origin at low and high redshift, 
it is caused by the fall-off in the probability distribution of the 
gas density at the onset of rapid, non-linear collapse. Simi- 
larly, one would expect the flattening observed at logNni <; 16 
at z ~ 3 to shift to logA^// <; 15 by z ~ 0, in agreement with the 
observations of Weymann et al. (1998). 

5. CONCLUSIONS 

In the last decade semi-analytic models and numerical simu- 
lations were used to show that the Lya forest is produced natu- 
rally in cold dark matter models with initially scale invariant, 
adiabatic fluctuations. Low column density absorption lines 
are predicted to arise in the smoothly fluctuating, photoionized 
IGM, which contains a large fraction of the baryons in the uni- 
verse. This paper demonstrates that this physical picture is in 
fact generic and can be derived using straightforward physical 
arguments, without making any specific assumptions about the 
cosmology, the presence and distribution of dark matter or the 
mechanism for structure formation. 

The key physical argument is that the sound crossing time 
(f sc ) along any sightline intersecting a cloud, is in general of 
order the local dynamical time (fd yn ), regardless of the shape 
of the cloud and regardless of whether the cloud as a whole 
is in dynamical equilibrium. This argument holds if the char- 
acteristic size (the scale over which the density is of order the 
maximum density) is smaller than the sound horizon, which 
is true for overdense absorbers (except for a short period after 
reionization). If, for whatever reason, a large departure from lo- 
cal hydrostatic equilibrium occurs, then local equilibrium will 
be restored on a timescale min(f sc ,fd yn ), which is much smaller 
than the Hubble time if the absorber is overdense and out of 
equilibrium. The only way to systematically violate the con- 
dition f sc ~ fdyn, is to postulate that all Lya forest absorbers 
are pressure confined, a proposal that has been ruled out on 
both observational and theoretical grounds (e.g., Rauch 1998), 
that they are rotationally supported and that their spin axes 
are always nearly perpendicular to the line of sight, or that 
the fraction of baryons in quasi-stable (f ~ f sc ~ fdyn < H~ l ) 
gas clouds is negligible compared with the fraction in transient 
(f ~ min(f sc ,f dy n) < max(f sc ,f dy n)) clouds. 

Using just the condition f sc ~ fd yn and an estimate of the 



temperature (T ~ 10 K) and ionization rate (T ~ 10 s -1 at 
z ~ 3), it was shown that unless the ratio of baryonic to col- 
lisionless matter is orders of magnitude lower than is gener- 
ally assumed, the Lya forest absorbers must be extended struc- 
tures of low overdensity. Furthermore, the conclusion that the 
low column density absorbers contain a large fraction of the 
baryons, follows directly from the observed column density dis- 
tribution. 

Scaling relations were derived for the radial sizes and char- 
acteristic densities of the absorbers. It was shown that if the 
parameters of the model are set to the values appropriate for the 
specific model that was simulated by Dave et al. (1999), then 
the density - column density relation agrees very well with their 
empirical fitting formula. Finally, the mass-weighted probabil- 
ity distribution of the gas density was derived from the observed 
column density distribution. This lead to several new insights: 
the shape of the column density distribution, in particular the 
deviations from a single power-law, and its evolution with red- 
shift, reflect the shape of the matter distribution, which agrees 
remarkably well with the shape of the distribution produced by 
the growth of structure in an expanding universe via gravita- 
tional instability. 

This work demonstrates that Jeans smoothing is central to 
the small-scale structure of the Lya forest. It is therefore likely 
that studies in which the effect of gas pressure is neglected, or 
in which the Jeans length is varied only in time, are only reli- 
able on scales greater than the sound horizon. On these scales 
(^> 10 2 kms" 1 ), gas pressure is unimportant and the gas should 
be a good tracer of the large-scale distribution of the matter. 

It is important to note that the observational support for the 
current physical picture of the Lya forest is much less strong for 
the high column density lines (Nhi > 10 16 cm" 2 ) than it is for 
the low column density forest, which formed the main focus of 
this work. It is for example possible, perhaps even likely, that 
galactic winds affect some of the high-density gas that gives 
rise to the strong absorption lines (e.g., Theuns, Mo, & Schaye 
2001). 

Although the derived properties of the Lya forest absorbers 
are not particularly sensitive to the presence of dark matter, dark 
matter may still be required to form the observed structure in 
the forest from the small-amplitude fluctuations (S ~ 10~ 5 ) ob- 
served at z ~ 10 3 in the cosmic microwave background. Indeed, 
it is well-known that if gravitational instability is responsible 
for structure formation, then dark matter (or a modification of 
gravity) is required to grow the observed (non-linear) structure. 
Here, the shape of the matter distribution function was derived 
directly from the observed column density distribution using 
the analytic model, which does not make any assumptions about 
the mechanism for structure formation. The fact that the derived 
shape agrees with the expectations from gravitational instability 
in an expanding universe, lends credence to the use of ab-initio 
hydrodynamical simulations to investigate the detailed proper- 
ties of the forest and their dependence on the parameters of the 
models. 

I am grateful to Tom Theuns for giving me permission to 
use one of his simulations for figure 2. It is a pleasure to 
thank Anthony Aguirre, John Bahcall, Jordi Miralda-Escude, 
and Eliot Quataert for discussions and a careful reading of the 
manuscript. This work was supported by a grant from the W. 
M. Keck Foundation. 
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APPENDIX 



FULL EQUATIONS 

In the equations appearing in the main text, numerical values were inserted for the natural constants, as well as for many other 
physical parameters. For reference, a list of the full expressions is listed below. The contributions to the gas and electron densities of 
elements other than hydrogen and helium are neglected. 



Hydrodynamics 



The dynamical timescale ?dyn = 1/ \[Gp is 
The sound crossing timescale f sc =L/c s is 
The condition fd yn = fsc defines the Jeans length, 



fdyn — 



Uc = L 



Gm H n H 



L j =\~^) l \l-Y)^ffn^l\ 



\pm 2 H GJ 



(Al) 



(A2) 



(A3) 



The gaseous Jeans mass M g j = p g L] is 



M, 



jk 



3/2 



4/3 „ 

v M m n G 



(A4) 



and the total Jeans mass is a factor l/f g greater. The Jeans column density in hydrogen Nhj = rtnLj is 

1/2 

\pm 2 H Gj 

Equations valid for an optically thin plasma 
The neutral fraction of a highly ionized, optically thin gas is 

n m l-Y/2(3 H ii 
n H l-Y T 



(A5) 



(A6) 
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Substituting this into the expression for the Jeans column density yields 



Nr 



The hydrogen number density is related to the density contrast 5 = (p- p)j p, 



-{\-Y){\+z)W + 5). 



Using this relation, the neutral hydrogen column density can be expressed in terms of the density contrast, 
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(A7) 
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Substituting equation A7 into equation A3 yields an expression relating the Jeans length to the neutral hydrogen column density, 



Lj= (^p) [(i-D(i-r/2)] 1/3 /f p l 'lT-y^i% 



1/3 
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(A10) 



Similarly, by substituting equation A7 into equation A4 the gaseous Jeans mass can be expressed in terms of Nhi, 
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M g ,j = 



4- [(l-F)(l-F/2)] 1/3 /|/ 3 /3^r-V3rV3</3, 
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(All) 



3H 

and the total Jeans mass is again a factor 1 J f g greater. The density of neutral hydrogen relative to the critical density p c = can 
be obtained from the number density of absorption lines per unit absorption distance and column density, f(Nm,Z) (see equation |l5| 
for the definition of f), 

8irG/ " // f N HI f(N HI ,z)dN HI . (A12) 



Qhi — 



3H c 



Using equations |A6| and |A7| , this can be converted into an expression for the total gas density, 

N£f(N m ,z)dN H i. 



8 3 H c\ p 



(A13) 



If the gas is not isothermal, then (3hii and T depend on Nhi and should be moved inside the integral. 

Finally, the results derived in this paper are equally valid for the He II Lya forest, provided that Nhi is replaced by Nneii using 



"Helll 1 HI 



N H eII _ 

Nhi ~4(1-F) (3 HI1 T He n 



(A14) 



where f3 He in/ Phii « 5.3. 



